Finding the optimal design of a product or a tool is a ubiquitous and recurring task across many engineering domains, e.g., including biomedical engineering, material engineering, and production engineering. While computer simulations have become an essential part of the workflow to tackle this problem, performing multiple simulations repeatedly within an optimization loop can become prohibitively expensive for realistic applications, with single simulation runs often requiring multiple days on modern high-performance computing architectures. This motivates the need for novel approaches to optimal design, e.g., via (scientific) machine learning techniques. Recently, Reinforcement Learning (RL) has gained interest as a learning-based approach to optimal design tasks in engineering. RL is based on trial-and-error interactions of an agent (a learning algorithm) with an environment (encoding a task that the agent is supposed to solve), thereby mimicking how humans learn to accomplish a new task. For each action, the agent obtains a reward (i.e., a numerical signal quantifying how good the undertaken action was for solving the task at hand) and information about the subsequent state of the environment. We can leverage this approach to reformulate classical optimization problem as learning tasks. The advantage over classical optimization formulations is that the agent learns a more general strategy, which makes this approach especially attractive if multiple similar optimization problems are supposed to be solved repeatedly. Building on previous work, the first part of the talk will focus on applying RL to optimize the shape of flow channels inspired by the manufacturing process of profile extrusion. We will compare two strategies for formulating the learning problem and the performance of different algorithms. In the second part of this talk, we aim to extend the proposed framework to solid mechanics and tackle the optimal design of fibers embedded into a material matrix for tailoring custom material responses.